Phoenix Evals runs LLM-as-judge on traces or datasets. Pre-built templates: hallucination, relevance, toxicity, QA. Outputs scored DataFrames.
Phoenix is an AI observability platform for tracing, evaluating, and debugging LLM apps. 9.1K+ stars. OpenTelemetry, evals, prompt management.
Phoenix instruments OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI via OpenInference. Local UI or Arize cloud. No per-call code changes.
%20by%20%E2%80%9ELisa%20Wischofsky%E2%80%9D%2C%20licensed%20under%20%E2%80%9ECC0%201.0%E2%80%9D%20(https%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F)%3C%2Fdc%3Arights%3E%3C%2Frdf%3ADescription%3E%3C%2Frdf%3ARDF%3E%3C%2Fmetadata%3E%3Cmask%20id%3D%22viewboxMask%22%3E%3Crect%20width%3D%22980%22%20height%3D%22980%22%20rx%3D%22490%22%20ry%3D%22490%22%20x%3D%220%22%20y%3D%220%22%20fill%3D%22%23fff%22%20%2F%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23viewboxMask)%22%3E%3Crect%20fill%3D%22url(%23backgroundLinear)%22%20width%3D%22980%22%20height%3D%22980%22%20x%3D%220%22%20y%3D%220%22%20%2F%3E%3Cdefs%3E%3ClinearGradient%20id%3D%22backgroundLinear%22%20gradientTransform%3D%22rotate(277%200.5%200.5)%22%3E%3Cstop%20stop-color%3D%22%23FEF3E2%22%2F%3E%3Cstop%20offset%3D%221%22%20stop-color%3D%22%23FFE4CC%22%2F%3E%3C%2FlinearGradient%3E%3C%2Fdefs%3E%3Cg%20transform%3D%22translate(10%20-60)%22%3E%3Cpath%20d%3D%22M495%20234c28%201%2056%206%2084%2015a201%20201%200%200%201%20100%2065c19%2025%2031%2055%2035%2086a361%20361%200%200%201%202%20101l-5%2034-4%2020c-1%202-1%202-3%202-2-4-2-8-2-12-1-15%201-29%202-44%201-24%203-48%201-72l-6-45c-7-28-20-54-40-75-18-18-40-33-63-43-22-9-44-15-67-19-18-3-36-4-54-4a277%20277%200%200%200-128%2029c-13%209-25%2020-35%2032a276%20276%200%200%200-55%20139c-1%2024%200%2047%208%2070l7%2015%2012%201%201%203c-3%202-7%202-11%203-10%202-20%206-29%2012-7%205-14%2011-18%2019-5%2010-3%2022-1%2033%205%2019%2017%2037%2033%2049%2011%209%2025%2014%2039%2016%2010%202%2020%200%2030-1%205%204%208%2012%2011%2017l18%2027a136%20136%200%200%200%2066%2050c22%209%2045%2014%2068%2019l41%208%2017%205v4h-12a535%20535%200%200%201-67-8%20255%20255%200%200%201-84-33c-11-7-21-16-29-27a214%20214%200%200%201-16%20123c-15%2027-40%2048-70%2057-3%201-7%202-10%201-2%200-2-1-2-3%201-3%204-5%206-6%2012-8%2026-14%2038-23%2013-11%2024-23%2031-38%206-15%2010-32%2012-49%202-13%203-26%202-40l-4-38-18-36-19%204a88%2088%200%200%201-75-38%2093%2093%200%200%201-20-58c0-9%203-18%208-25s12-13%2019-17c5-4%2012-6%2018-8l-9-24c-5-22-5-45-2-67a292%20292%200%200%201%2063-149c9-10%2018-19%2029-26%207-4%2014-5%2020-9%2043-19%2090-25%20137-22Z%22%20fill%3D%22%23000%22%2F%3E%3Cpath%20d%3D%22M475%20243a276%20276%200%200%201%20121%2023c23%2010%2045%2025%2063%2043%2020%2021%2033%2047%2040%2075l6%2045c2%2024%200%2048-1%2072-1%2015-3%2029-2%2044%200%204%200%208%202%2012l-1%2020c-2%205-2%2010-2%2016-2%2024-2%2049-6%2074-1%2012-3%2025-8%2036-3%209-8%2017-15%2024-12%2013-27%2024-43%2033-19%2011-39%2016-58%2024-5%202-10%204-13%208%208%201%2015%200%2023-2-5%2012-6%2024-6%2036%200%205-1%2011%201%2016%203%205%208%209%2013%2012l35%2013c22%208%2043%2018%2065%2027%202%204%205%207%206%2012%206%2011%207%2023%203%2035-5%2013-15%2024-27%2031-16%2011-36%2019-55%2024a494%20494%200%200%201-235-7c-20-7-40-14-58-24-14-8-29-18-40-31-6-8-11-19-12-29a122%20122%200%200%200%2086-107c4-24%204-49%200-73%208%2011%2018%2020%2029%2027l27%2014a331%20331%200%200%200%20124%2027h12v-4l-17-5-41-8c-23-5-46-10-68-19-13-6-27-13-39-21-10-9-19-18-27-29l-18-27c-3-5-6-13-11-17-10%201-20%203-30%201-14-2-28-7-39-16a94%2094%200%200%201-33-49c-2-11-4-23%201-33%204-8%2011-14%2018-19%209-6%2019-10%2029-12%204-1%208-1%2011-3l-1-3-12-1-7-15c-8-23-9-46-7-70a276%20276%200%200%201%2054-139c10-12%2022-23%2035-32a277%20277%200%200%201%20128-30Z%22%20fill%3D%22%23ffffff%22%2F%3E%3Cpath%20d%3D%22M260%20571c4%200%207%200%2011%202%205%201%2011%204%2014%208%202%203%202%207%200%2010-5%206-12%2010-16%2016%206%204%2014%206%2021%209%204%201%208%203%2011%206l1%203-8-2-24-7c-4-1-8-3-9-7-1-3%201-7%203-10l15-13c-4-7-15-8-19-15ZM703%20577l2%203%202%2016a483%20483%200%200%201-2%2089c-2%2011-5%2023-10%2033a198%20198%200%200%201-106%2071c-1%2014-3%2031-2%2047l5%205c8%206%2018%209%2027%2012a451%20451%200%200%201%2068%2028c7%204%2013%209%2018%2015-5%201-10-1-16-2-22-9-43-19-65-27l-35-13c-5-3-10-7-13-12-2-5-1-11-1-16%200-12%201-24%206-36-8%202-15%203-23%202%203-4%208-6%2013-8%2019-8%2039-13%2058-24%2016-9%2031-20%2043-33%207-7%2012-15%2015-24%205-11%207-24%208-36%204-25%204-50%206-74%200-6%200-11%202-16Z%22%20fill%3D%22%23000%22%2F%3E%3Cpath%20d%3D%22M681%20495c3%200%207%200%2010%202%202%201%202%202%201%204l-9%203-24%205c-1%206-1%2013-4%2019-2%205-6%209-11%209s-9-3-11-7c-3-5-5-11-6-18-4%200-9%201-12-1-1-3%203-4%205-4%2012-5%2026-7%2039-9%207-2%2015-2%2022-3ZM496%20502c5%200%2011%200%2016%203l3%203c-7%202-13%204-20%204-2%209-4%2017-9%2025-2%203-5%206-9%205-5-1-10-4-13-9-4-5-8-11-8-18l-56%202c-2-1-6%200-6-3%203-3%208-3%2012-4l33-4c19-2%2038-4%2057-4Z%22%20fill%3D%22%23000000%22%2F%3E%3Cpath%20d%3D%22m650%20510-6%2017c-4-4-6-10-9-15l15-2ZM484%20513l-6%2018-4-2c-5-4-8-10-12-15l18-1h4Z%22%20fill%3D%22%23fff%22%2F%3E%3Cg%20fill%3D%22%23000000%22%3E%3Cpath%20d%3D%22M675%20381c5%200%208%204%209%208-11%203-23%208-34%2013l-16%208c-7%204-14%207-22%209-5%201-9%200-13%203l-7%208c-3%203-4%207-6%2012-1-1-3-2-3-4-2-5%200-11%202-16%202-6%203-12%208-17%203-4%205-8%209-10%204-3%2010-4%2014%200l1%206%2025-14c11-4%2021-8%2033-6ZM481%20383c14%202%2032%201%2044%207%209%204%2013%2011%2018%2019l-1%204c-4%203-9%204-14%202-4-2-6-6-9-8-2-2-5-2-8-3l-25-2c-12-1-24-6-36-7l-27-5-5-2c9-3%2019-4%2029-5%2011%200%2023-2%2034%200Z%22%2F%3E%3Cpath%20d%3D%22M540%20387c6-2%2012-2%2016%202%206%203%209%209%2011%2015%200-4%202-9%205-12%206%203%2011%209%2013%2015%201%205%202%2010%200%2015s-9%208-13%205c-4-4-4-12-5-18-1%204-2%208-5%2010s-7%201-9-2l-5-12-7-15-1-3Z%22%2F%3E%3C%2Fg%3E%3Cpath%20d%3D%22M594%20572a37%2037%200%200%201%2019%2029c0%209-5%2018-13%2022l-17%207c-4%201-6%202-9%200-2-3-1-4%201-6%206-5%2015-7%2021-12%203-2%205-5%205-9%200-10-5-17-7-26v-5Z%22%20fill%3D%22%23000000%22%2F%3E%3Cpath%20fill-rule%3D%22evenodd%22%20clip-rule%3D%22evenodd%22%20d%3D%22M438%20634c-4%206-11%2011-19%2013%200-4%203-7%206-9%204-3%208-5%2013-4Zm14%2012%2032%205%2024%206c15%204%2029%208%2044%2010%2010%201%2020%200%2029-1%2012-2%2023-4%2036-1%206%202%2011%206%2015%2011%203%205%204%2010%202%2016s-7%2013-13%2017c-7%205-15%206-24%207l-30%203-29-2-21-5-10-3-12-3-14-4-23-8-17-10c-7-5-10-15-8-23%202-9%2010-15%2019-15Zm4%209-9%201c-4%201-7%204-8%208%200%203%202%206%203%208v1l8%208%204%202c0-9%200-19%204-27l-2-1Zm9%202a703%20703%200%200%201%2040%2010c-2%2011-5%2022-4%2033l-22-6-14-5-4-2%204-17v-13Zm87%2020c-14-1-27-5-41-8a46%2046%200%200%201-1%2018c-1%205-2%2010-5%2014l9%202%2013%204%2017%203%205%201c-3-10%200-21%202-30l1-4Zm54-5c4%200%208%200%2012%202%203%201%206%205%208%208%201%202%202%205%201%208-3%205-7%2010-12%2013s-10%204-16%205l-2%201%207-18%202-19Zm-13%2025%205-25-15%203-23%202c0%2010-2%2019-5%2029l-2%205a135%20135%200%200%200%2035-1l4-1%201-12Z%22%20fill%3D%22%23000000%22%2F%3E%3Cpath%20d%3D%22M631%20178a213%20213%200%200%201%2054%2022c3%202%203%205%204%208l-13-2c-10-1-20-3-30-2-22%201-43%206-62%2016%2016%200%2031%200%2047%204%2029%206%2056%2018%2077%2039%2017%2016%2030%2036%2036%2059%205%2019%206%2039%201%2059a127%20127%200%200%201-73%2084l-20%209-5-2c1-10%204-18%206-27l2-24c0-11-2-22-4-33l-9%2020c-9%2017-24%2032-41%2041-12%208-27%2013-41%2017-18%205-37%207-56%209h-19c-3%200-3-2-5-4l7-10c5-9%2011-18%2015-28l7-25c4-14%205-28%205-43a299%20299%200%200%201-108%20125c-12%208-26%2013-39%2018-6%202-11%204-17%201l2-13%204-30c-7%2015-13%2032-16%2049-5%2028-1%2055%205%2083l2%2015c-8%202-16%201-24-2-26-10-49-32-59-59%200-4-4-4-6-6-10-5-18-14-23-23-9-14-15-31-17-48a204%20204%200%200%201%2030-136c11-20%2025-37%2041-53a146%20146%200%200%201%2075-38%20432%20432%200%200%200%2057-16c27-10%2051-24%2077-36a229%20229%200%200%201%20133-18Z%22%20fill%3D%22%23000000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(10%20-60)%22%3E%3C%2Fg%3E%3C%2Fg%3E%3C%2Fsvg%3E)